The Paradox of Overfitting
نویسنده
چکیده
Preface The Faculty of Artificial Intelligence in Groningen does research on the technical aspects of cognition: reasoning, navigation, communication and learning. A technical approach requires its representations and algorithms to be robust and easy to use. It adds a crucial constraint to its domain of research: that of complexity. But complexity is more than a prominent constraint on real world applications. In recent years complexity theory has developed into an exciting area of mathematical research. It is a powerful mathematical tool that can lead to surprising solutions where conventional methods come to a dead end. Any theory of cognition that dismisses complexity constraints for the sake of theoretical freedom misses a powerful mathematical ally. This is why I have chosen complexity theory as the theme of my master's thesis, and specifically the experimental evaluation of an application of complexity theory on learning algorithms. Applied sciences like medicine, physics or chemistry spend immense sums of money on technologies that visualize the objects of their research. Progress is published in all the media formats available. First of all this is done to get a better grip on the complicated problems of the science. But with such a wealth of information even a lay person finds it relatively easy to understand the synthesis of a virus, a special type of brain damage or the problem of bundling plasma streams in nuclear fusion. In statistics and complexity theory tools for visualization are rare and multi media publications are unheard of. This is not without consequences as the general public has almost no idea of statistics and complexity theory. The Nobel prize economy 2002 was shared by Daniel Kahneman for his findings on the sort of statistical awareness that actually governs our stock markets [KST82]. Though the rules of thumb that real-world decision-makers use are strong, they do not reflect statistical insight. To help the individual, whether scientist or not, to understand the theory in question I have tried to fit it into a modern graphical user interface. This document is written in pdf-L A T E X. It contains colored images and hyper-links that can best be accessed with a pdf-viewer like acroread. Together with the application and other information it is online available at
منابع مشابه
An Approach to Reducing Overfitting in FCM with Evolutionary Optimization
Fuzzy clustering methods are conveniently employed in constructing a fuzzy model of a system, but they need to tune some parameters. In this research, FCM is chosen for fuzzy clustering. Parameters such as the number of clusters and the value of fuzzifier significantly influence the extent of generalization of the fuzzy model. These two parameters require tuning to reduce the overfitting in the...
متن کاملA Short Introduction to Model Selection, Kolmogorov Complexity and Minimum Description Length (MDL)
The concept of overfitting in model selection is explained and demonstrated. After providing some background information on information theory and Kolmogorov complexity, we provide a short explanation of Minimum Description Length and error minimization. We conclude with a discussion of the typical features of overfitting in model selection. 1 The paradox of overfitting Machine learning is the ...
متن کاملBertrand’s Paradox Revisited: More Lessons about that Ambiguous Word, Random
The Bertrand paradox question is: “Consider a unit-radius circle for which the length of a side of an inscribed equilateral triangle equals 3 . Determine the probability that the length of a ‘random’ chord of a unit-radius circle has length greater than 3 .” Bertrand derived three different ‘correct’ answers, the correctness depending on interpretation of the word, random. Here we employ geomet...
متن کاملParadox in the Poetry of Hazin-e Lahiji
Paradox is one of the literary techniques in the poetry of the Safavid poets. Hazin-e Lahiji, like so many other poets of that age, employed this technique in his pursuit and showed that "unfamiliar meaning". Paradox is used in the poetry of Hazin-e Lahiji for the purpose of defamiliarization and exoticism. The poet in order to create new implications and subtle and insightful points and also t...
متن کاملThe Paradox of Health Policy: Revealing the True Colours of This ‘Chameleon Concept’; Comment on “The Politics and Analytics of Health Policy”
Health policy has been termed a ‘chameleon concept’, referring to its ability to take on different forms of disciplinarity as well as different roles and functions. This paper extends Paton’s analysis by exploring the paradox of health policy as a field of academic inquiry—sitting across many of the boundaries of social science but also marginalised by them. It situates contemporary approaches ...
متن کامل